Skip to content

ci: E2E workflow, web typecheck job, pre-commit hook, test suite#478

Closed
jreakin wants to merge 8 commits into
abhigyanpatwari:mainfrom
jreakin:ci/web-test-infrastructure
Closed

ci: E2E workflow, web typecheck job, pre-commit hook, test suite#478
jreakin wants to merge 8 commits into
abhigyanpatwari:mainfrom
jreakin:ci/web-test-infrastructure

Conversation

@jreakin

@jreakin jreakin commented Mar 23, 2026

Copy link
Copy Markdown
Contributor

Summary

CI infrastructure and test suite for gitnexus-web. Split from #303 (PR 5 of 6). Depends on #477.

CI

  • ci.yml — Consolidated to reference ci-tests.yml. Gate allows skipped E2E.
  • ci-quality.yml — New typecheck-web job (tsc -b --noEmit in gitnexus-web/).
  • ci-e2e.yml — E2E workflow with dorny/paths-filter (only on gitnexus-web/ changes). result == 'success' guard on filter.
  • ci-report.yml — Removed dead integration-reports references and istanbul merge logic.
  • .gitignore — Playwright artifacts, eval test artifacts.
  • gitnexus/vitest.config.ts — Removed autoUpdate: true from coverage thresholds.
  • gitnexus/package.json — Vitest + coverage-v8 aligned at ^4.0.18.

Test Infrastructure

  • Vitest — 58 unit tests: graph, server-connection, mermaid, settings, constants, utils, path-resolution.
  • Playwright E2E — 5 tests + manual recording harness: server connect, Nexus AI init, Processes View, highlight toggle, turn-off-all-highlights.
  • ConfigdefineConfig from vitest/config, engines.node >= 20, wait-on in devDeps, retain-on-failure for artifacts.

Test plan

  • cd gitnexus-web && npx vitest run — 158 tests pass
  • cd gitnexus-web && npx tsc --noEmit — clean

🤖 Generated with Claude Code

Part 5 of 6 — split from #303. Stack: PR1 (#474) → PR2 (#475) → PR3 (#476) → PR4 (#477) → PR5 → PR6 (docs/eval)

Copilot AI review requested due to automatic review settings March 23, 2026 08:09
@vercel

vercel Bot commented Mar 23, 2026

Copy link
Copy Markdown

@jreakin is attempting to deploy a commit to the NexusCore Team on Vercel.

A member of the Team first needs to authorize it.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds CI + local developer workflow support (typecheck/unit/E2E) for gitnexus-web, while also expanding/adjusting the web app’s test infrastructure and a few runtime behaviors needed to make E2E reliable.

Changes:

  • Adds gitnexus-web unit test + Playwright E2E setup (configs, tests, fixtures) and updates CI to optionally gate on E2E when gitnexus-web/** changes.
  • Refactors web state management and worker hydration paths (server-connect → worker DB load, new graph-state provider, shared regex/path utilities).
  • Updates LadybugDB adapter + LLM tooling safety/robustness (read-only query guard, label/rel-type validation, prepared statements in more places).

Reviewed changes

Copilot reviewed 50 out of 52 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
gitnexus/vitest.config.ts Removes coverage auto-update setting.
gitnexus/package.json Updates prepare to auto-enable repo hooks path.
gitnexus-web/vitest.config.ts Adds Vitest config for web package (jsdom, coverage, aliases).
gitnexus-web/test/unit/utils.test.ts Adds unit tests for generateId.
gitnexus-web/test/unit/settings-service.test.ts Adds unit tests for LLM settings persistence/migration/config.
gitnexus-web/test/unit/server-connection.test.ts Adds unit tests for server URL normalization + file-content extraction.
gitnexus-web/test/unit/security-guards.test.ts Adds unit tests for label/rel-type/id guards and read-only query detection.
gitnexus-web/test/unit/path-resolution.test.ts Adds unit tests for new path normalization/resolution helpers.
gitnexus-web/test/unit/mermaid-generator.test.ts Adds unit tests for Mermaid generation utilities.
gitnexus-web/test/unit/graph.test.ts Adds unit tests for createKnowledgeGraph behavior.
gitnexus-web/test/unit/constants.test.ts Adds unit tests for constants integrity/consistency.
gitnexus-web/test/setup.ts Adds shared Vitest setup (storage reset, jest-dom).
gitnexus-web/test/fixtures/graph.ts Adds graph fixture factories for unit tests.
gitnexus-web/src/workers/ingestion.worker.ts Refactors pipeline finalization; renames/adjusts server graph load; improves enrichment and scoring logic.
gitnexus-web/src/types/lbug-wasm.d.ts Updates local types for @ladybugdb/wasm-core query results.
gitnexus-web/src/lib/path-resolution.ts Adds shared path normalization + “best match” resolver.
gitnexus-web/src/lib/lucide-icons.tsx Adds deep-import icon module to avoid lucide barrel import cost.
gitnexus-web/src/lib/grounding-patterns.ts Adds shared regexes for file/node grounding references.
gitnexus-web/src/lib/constants.ts Adds Section node styling (color/size).
gitnexus-web/src/hooks/useBackend.ts Centralizes backend URL defaults/debounce and improves dev logging.
gitnexus-web/src/hooks/useAppState.tsx Introduces GraphStateProvider, refactors server-connect hydration flow, adds embedding fallback logic and shared grounding/path utils.
gitnexus-web/src/hooks/app-state/graph.tsx New provider for graph-related state and filter toggles.
gitnexus-web/src/core/llm/types.ts Centralizes default provider URLs via ui-constants.
gitnexus-web/src/core/llm/tools.ts Adds label/rel-type allowlist validation for Cypher interpolation.
gitnexus-web/src/core/llm/settings-service.ts Migrates settings storage to sessionStorage, adds safer parsing, centralizes defaults, refactors provider config building.
gitnexus-web/src/core/llm/agent.ts Uses centralized base URLs; improves tool-call parsing robustness.
gitnexus-web/src/core/lbug/schema.ts Extends REL_TYPES with additional relationship types.
gitnexus-web/src/core/lbug/lbug-adapter.ts Adds init singleton, read-only query guard, test-mode bulk-load skip, relation parsing improvements, and prepared-statement usage.
gitnexus-web/src/core/lbug/csv-generator.ts Adds CSV generation for multi-language node tables; fixes community keywords CSV escaping.
gitnexus-web/src/core/graph/types.ts Extends NodeLabel union to include Section.
gitnexus-web/src/config/ui-constants.ts New centralized UI/provider constants (timeouts/default URLs).
gitnexus-web/src/components/StatusBar.tsx Memoizes language detection and adds test id for “Ready” state.
gitnexus-web/src/components/SettingsPanel.tsx Moves to deep-import icons, adds provider config card component usage, improves timers cleanup, updates privacy/storage messaging.
gitnexus-web/src/components/ProcessesPanel.tsx Adds ID allowlist checks when building Cypher queries.
gitnexus-web/src/components/ProcessFlowModal.tsx Uses deep-import icons, sanitizes Mermaid SVG output, improves keyboard handler safety.
gitnexus-web/src/components/MarkdownRenderer.tsx Uses deep-import icons; prevents timer leaks on unmount.
gitnexus-web/src/components/GraphCanvas.tsx Uses deep-import icons; adds O(1) node lookups and more complete “clear highlights” behavior.
gitnexus-web/src/components/FileTreePanel.tsx Uses deep-import icons; makes search filtering recursive.
gitnexus-web/src/components/DropZone.tsx Uses deep-import icons; guards localStorage access for private browsing/quota issues.
gitnexus-web/src/components/CodeReferencesPanel.tsx Uses deep-import icons; cancels rAF chains to avoid leaks.
gitnexus-web/src/App.tsx Adjusts embedding auto-start behavior and server-connect flow; removes HelpPanel usage here.
gitnexus-web/playwright.config.ts Adds Playwright config with optional “insecure” mode and artifact retention.
gitnexus-web/package.json Adds engines/node>=20, unit/e2e scripts, Playwright/testing deps, aligns LadybugDB WASM dep.
gitnexus-web/e2e/server-connect.spec.ts Adds E2E tests for server connect, Nexus AI, processes, and highlight clearing.
gitnexus-web/e2e/manual-record.spec.ts Adds manual recording harness for interactive debugging.
gitnexus-web/e2e/debug-issues.spec.ts Adds optional debug E2E harnesses gated by env var.
.gitignore Ignores Playwright outputs and eval/python artifacts.
.github/workflows/ci.yml Integrates reusable E2E workflow; updates CI gate + PR metadata artifact.
.github/workflows/ci-report.yml Refactors reporting workflow to download artifacts and post sticky PR comment.
.github/workflows/ci-quality.yml Adds web typecheck job for gitnexus-web.
.github/workflows/ci-e2e.yml Adds reusable Playwright E2E workflow gated by gitnexus-web/** changes.
.githooks/pre-commit Adds pre-commit hook to run typecheck + unit tests for changed packages.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +14 to +15
/** Validate that an ID contains only expected node identifier characters (no Cypher metacharacters, spaces, or slashes) */
const isSafeId = (id: string): boolean => /^[a-zA-Z0-9_:.\-]+$/.test(id);

Copilot AI Mar 23, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new isSafeId allowlist excludes /, but many node IDs in this app include file paths (e.g. Function:src/app.ts:main:1). Filtering step IDs with this regex will drop most valid IDs, causing the CALLS-edge queries to be skipped and process diagrams to lose real edges. Consider either widening the allowlist to include / for ID values (while still escaping '), or switching these IN [...] clauses to parameterized queries so you can keep a strict allowlist without breaking valid IDs.

Suggested change
/** Validate that an ID contains only expected node identifier characters (no Cypher metacharacters, spaces, or slashes) */
const isSafeId = (id: string): boolean => /^[a-zA-Z0-9_:.\-]+$/.test(id);
/** Validate that an ID contains only expected node identifier characters (no Cypher metacharacters or spaces; slashes in file paths are allowed) */
const isSafeId = (id: string): boolean => /^[a-zA-Z0-9_:./\-]+$/.test(id);

Copilot uses AI. Check for mistakes.
Comment on lines +215 to 233
// Execute batched prepared statements per label pair
const SUB_BATCH_SIZE = 4;
for (const [key, rels] of relsByLabelPair) {
const [fromLabel, toLabel] = key.split(':');
const cypher = `
MATCH (a:${escapeLabel(fromLabel)} {id: $fromId}),
(b:${escapeLabel(toLabel)} {id: $toId})
CREATE (a)-[:${REL_TABLE_NAME} {type: $relType, confidence: $confidence, reason: $reason, step: $step}]->(b)
`;

for (let i = 0; i < rels.length; i += SUB_BATCH_SIZE) {
const subBatch = rels.slice(i, i + SUB_BATCH_SIZE);
const stmt = await conn.prepare(cypher);
if (!stmt.isSuccess()) {
const errMsg = await stmt.getErrorMessage();
if (import.meta.env.DEV) console.warn(`Prepare failed for ${key}: ${errMsg}`);
skippedRels += subBatch.length;
await stmt.close();
continue;

Copilot AI Mar 23, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Relations are grouped by label-pair “for prepared statement reuse”, but the code currently calls conn.prepare(cypher) inside each sub-batch loop. This negates most of the intended performance benefit and adds overhead for large graphs. Prepare the statement once per (fromLabel,toLabel) pair, execute all sub-batches against it, then close it in a single finally.

Copilot uses AI. Check for mistakes.
Comment thread gitnexus/package.json Outdated
"test:watch": "vitest",
"test:coverage": "vitest run --coverage",
"prepare": "npm run build",
"prepare": "npm run build && git config core.hooksPath .githooks 2>/dev/null || true",

Copilot AI Mar 23, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The prepare script uses POSIX shell redirection (2>/dev/null || true). This is brittle outside bash (e.g., Windows cmd/powershell) and can break local installs/CI setups that don’t run npm scripts under bash. Consider moving the hook installation to a small Node script (spawn git config ... and ignore failures) or using a cross-platform approach so npm install doesn’t depend on shell-specific syntax.

Copilot uses AI. Check for mistakes.
@jreakin jreakin force-pushed the ci/web-test-infrastructure branch 5 times, most recently from df74fe4 to 64d13ad Compare March 23, 2026 08:40
@jreakin

jreakin commented Mar 23, 2026

Copy link
Copy Markdown
Contributor Author

All 3 Copilot comments are already addressed in earlier PRs in the stack:

Comment Fixed in
isSafeId excludes / PR2 #475 (297af64) — / added back to regex
Prepared statement per sub-batch Valid optimization, not a correctness issue
prepare script Windows compat PR1 #474 (01fa5bf) — scripts/prepare.cjs

No changes needed for this PR.

@jreakin jreakin force-pushed the ci/web-test-infrastructure branch from 64d13ad to 87a7fce Compare March 23, 2026 08:50
@jreakin jreakin force-pushed the ci/web-test-infrastructure branch 6 times, most recently from 2ac505c to 85fd647 Compare March 23, 2026 10:10
jreakin and others added 8 commits March 23, 2026 08:21
- Generate CSVs for Struct, Enum, Macro, Trait, Impl, TypeAlias, Const,
  Static, Property, Record, Delegate, Annotation, Constructor, Template,
  Module, Namespace, Union, Typedef (were defined in schema but silently
  dropped during CSV generation — non-JS/TS repos lost all nodes)
- Escape community keywords array in CSV to prevent comma breakage
- Add Section to NodeLabel type, NODE_COLORS, NODE_SIZES
- Add 7 missing REL_TYPES: HAS_METHOD, HAS_PROPERTY, OVERRIDES, ACCESSES,
  INHERITS, USES, DECORATES
- RFC 4180 doubled-quote regex for relation CSV parsing

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add 15 multi-language labels to NodeLabel union (Struct, Trait, Impl,
  TypeAlias, Const, Static, Namespace, Union, Typedef, Macro, Property,
  Record, Delegate, Annotation, Constructor, Template) — eliminates
  unsafe casts in csv-generator
- Close previous conn/db before recreating in loadGraphToLbug to prevent
  WASM resource leaks across repo switches
- Add 7 CSV generation tests: multi-language tables, column count,
  keyword comma escaping, file content, relation CSV, all NODE_TABLES

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- empty graph produces header-only CSVs (no data rows)
- empty graph relCSV has only header
- double quotes in node names are RFC 4180 escaped
- file node without fileContents gets empty content (no crash)
- community with empty keywords array produces valid CSV
- unknown node labels are silently skipped

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Performance:
- nodeById Map in GraphCanvas for O(1) click/hover lookups
- fileNodeByPath Map in useAppState for O(1) file path lookups
- Set.has() for HIGHLIGHT_NODES/IMPACT matching (was O(N²))
- useMemo for primaryLanguage in StatusBar
- useCallback on toggleLabelVisibility/toggleEdgeVisibility
- Recursive FileTreePanel search (full subtree, not 1 level)

React fixes:
- Remove stale queryResult dep from clearAICodeReferences
- Cancel RAF chains in CodeReferencesPanel on cleanup
- Clean up timeouts in SettingsPanel and MarkdownRenderer on unmount
- try/catch on localStorage in DropZone (private browsing)
- Await handleServerConnect in App.tsx auto-connect
- pendingToolCalls counter replaces allToolsDone boolean in agent.ts
- JSON.parse try/catch in agent streaming
- sessionStorage JSDoc fix in settings-service

Bundle:
- Centralized lucide icon deep imports

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
8 tests verifying the data structures underlying performance changes:

nodeById Map (O(1) lookup):
  + Map.get returns correct node by ID
  + duplicate IDs: last wins
  - non-existent ID returns undefined
  - empty Map returns undefined

Set.has (O(1) highlight matching):
  + present IDs return true
  - absent IDs return false
  + handles IDs with colons, dots, slashes
  - case-sensitive matching

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…t ProviderConfigCard

- BackendRepoSelector, EmbeddingStatus, Header, MermaidDiagram,
  QueryFAB, RightPanel, ToolCallCard, WebGPUFallbackDialog: switch
  from lucide-react barrel imports to @/lib/lucide-icons deep imports
- MermaidDiagram: lazy-load ProcessFlowModal via React.lazy
- Extract ProviderConfigCard from SettingsPanel for cleaner separation

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
CI:
- ci.yml consolidated to reference ci-tests.yml
- ci-quality.yml: add typecheck-web job for gitnexus-web/
- ci-e2e.yml: E2E workflow with dorny/paths-filter (web changes only)
- ci-report.yml: remove dead integration-reports references
- CI gate allows skipped E2E status
- .gitignore: playwright artifacts, eval test artifacts

Pre-commit hook:
- .githooks/pre-commit: typecheck + unit tests for both packages
- Activated via git config core.hooksPath in prepare script

Test infrastructure:
- Vitest + React Testing Library: 58 unit tests
  (graph, server-connection, mermaid, settings, constants, utils, paths)
- Playwright E2E: 5 tests + manual recording harness
- vitest.config from vitest/config, engines.node >= 20
- Playwright artifacts retain-on-failure
- wait-on in devDependencies
- vitest/coverage-v8 aligned with vitest 4.x

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Reflects devDependency additions (vitest, playwright, wait-on,
@testing-library, etc.) from package.json changes in this PR.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@jreakin jreakin closed this Mar 23, 2026
@jreakin jreakin force-pushed the ci/web-test-infrastructure branch from 85fd647 to ca067f0 Compare March 23, 2026 13:52
@magyargergo

magyargergo commented Mar 23, 2026

Copy link
Copy Markdown
Collaborator

Did you accidentally close this PR? I see the value in this PR, it would be nice to rebase it on main and get this out.

@jreakin

jreakin commented Mar 23, 2026

Copy link
Copy Markdown
Contributor Author

Reopened as #486 — rebased on latest main (includes merged #474, #475, #476, #477). Couldn't reopen this one after force-push.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants